1. Technical Field
The present invention relates to an obstacle detection system for detecting an obstacle existing on a ground, such as a preceding vehicle, a pedestrian or a parked vehicle with cameras mounted on a vehicle, so as to support the safe drive of an automobile.
2. Background Art
The technology for detecting an obstacle with a sensor is coarsely divided into one utilizing a laser beam or ultrasonic waves and one utilizing TV cameras. The technology utilizing the laser beam costs high and is impractical. On the other hand, the technology utilizing the ultrasonic waves has a low resolution so that it is troubled by the detection precision of the obstacle.
On the contrary, the TV cameras are relatively inexpensive so that they are suitable for the obstacle detection from the aspects of the resolution, the measuring precision and the measuring range. In the case of using the TV cameras, there are methods of employing one camera and a plurality of cameras (or stereo cameras).
In the method of employing the single camera, the ground region and the obstacle region are separated with the clue of the informations such as the intensities, colors or textures of one image taken by the camera. For example, the intermediate intensity region of a low chroma, i.e., a gray region is extracted from the image to determine the ground region or a region of few textures so that the ground region is extracted while leaving the remaining region as the obstacle region. However, there are many obstacles having intensities, colors or textures similar to those of the ground. Therefore, this method finds it difficult to separate the obstacle region and the ground region under the general situations.
On the contrary, the method using a plurality of cameras detects the obstacle with a clue to three-dimensional informations. The technology for obtaining the three-dimensional informations of an object scene by using the plurality of cameras is generally called the “stereo vision”. According to this stereo vision, given corresponding points between stereo images, it is possible to determine the three dimensional position. If the positions and orientations of the individual cameras with respect to the ground plane are predetermined, the height of an arbitrary point in the images from the ground plane can be obtained by the stereo vision. Depending upon the presence or absence of the height, therefore, it is possible to separate the obstacle region and the ground region. It is difficult for the method using the single camera to detect the region having intensities, colors and textures similar to those of the ground, as the obstacle. According to the stereo vision, however, the obstacle is detected with a clue to the height from the ground plane so that the obstacle detection can be made in a more general scene.
The ordinary stereo vision is a technology for determining the distances of an arbitrary point on the image from the stereo cameras. For this technology, it is necessary to determine parameters in advance on the spacing and directions of the plurality of cameras and the focal lengths and the principal points of the camera lenses. The work for determining the parameters is called the “calibration”. For this calibration, there are prepared a number of points, the three-dimensional locations of which are known. The projected locations of the points on the images are determined to compute the parameters on the locations and positions of the cameras and the focal lengths of the camera lenses. However, these operations require a long time and many works to obstruct the practical obstacle detection by the stereo vision.
If it is sufficient to separate the ground region and the obstacle region on the images, however, the calibrations are not necessarily required. If the projected points of a point of the ground plane on the left and right images are designated by (ul, vl) and (ur, vr), the following relation holds:
                                                                                          u                  r                                =                                                                                                    h                        11                                            ⁢                                              u                        l                                                              +                                                                  h                        12                                            ⁢                                              υ                        l                                                              +                                          h                      13                                                                                                                          h                        31                                            ⁢                                              u                        l                                                              +                                                                  h                        32                                            ⁢                                              υ                        l                                                              +                                          h                      33                                                                                  ,                                                                                                                                    υ                r                            =                                                                                          h                      21                                        ⁢                                          u                      l                                                        +                                                            h                      22                                        ⁢                                          υ                      l                                                        +                                      h                    23                                                                                                              h                      31                                        ⁢                                          u                      l                                                        +                                                            h                      32                                        ⁢                                          υ                      l                                                        +                                      h                    33                                                                                                          (        1        )            h=(h11, h12, h13, h21, h22, h23, h31, h32, h33)T (T designates a transposition symbol) are parameters depending upon the locations and positions of the individual cameras with respect to the ground plane and upon the focal lengths and image origins of the lenses of the individual cameras. The parameters h are predetermined from the projected points of four or more points of the ground plane on the left and right images. By using these relations, the corresponding point P′ (ur, vr) on the right image is determined when it is assumed that an arbitrary point P(ul, vl) on the left image is present on the ground plane.
If the point P is present on the ground plane, the points P and P′ are a set of the correct corresponding points so that the difference between their intensities becomes small. When the points P and P′ have largely different intensities, therefore, it is decided that the point P belongs to the obstacle region. In the following, Equation 1 will be called the “ground plane constraint”.
In this method, the search for corresponding points is also unnecessary. The ordinary stereo method requires the search for matching points between the left and right images so that its computation cost is high because the correspondence is made by the search computation. However, the aforementioned method requires no correspondence search so that the computational cost is extremely inexpensive.
If the stereo cameras are fixed in the three-dimensional space, the obstacle existing on the ground plane can be detected by the parameters H once determined. While the vehicle is running, however, the relative geometric relationship between the ground plane and the individual cameras are changed time after time by the vibration of the vehicle itself and the change in the inclination of the ground. In short, the parameters h change during the traveling so that the ground plane constraint determined at a still time cannot be used for the obstacle detections during the traveling.
In order to solve this problem, there has been usually used a method for detecting the obstacle by computing the ground plane constraint using a number of featuring points (e.g., the corner points of the paints on the ground) on the ground plane. It is, however, difficult to extract the numerous featuring points on the ground plane, and it frequently occurs that the featuring points on the obstacle are erroneously extracted. Moreover, the correspondence search of the extracted featuring points has to be performed to raise the computation cost. Also, with the large number of parameters to be determined, there has been a problem that it is seriously difficult to determine the ground plane constraint stably.